March 11, 2024

Intense longitudinal data (ILD)

Uniquely suited to provide detailed information on psychological and behavioral processes over time at the within-person level

  • Daily Diary
  • Ecological momentary assessment (EMA)
  • Experience sampling method (ESM)
  • Ambulatory assessments (AA)
  • Sensors

54 publications indexed on PubMed in 2000, rising to 984 and 1203 publications in 2020 and 2021.





Empirical example I

Capturing (suicidal) crisis dynamics

Capturing crisis dynamics using ESM data


  • 26 patients, 60 observations per patient per CAB factor

Analyzing ILD data

Most popular statistical models for (emotion) time series:

  • the (multilevel) vector autoregressive (VAR) model
  • multilevel linear regression model

Statistical concern: multimodality and skewness are highly prevalent: “less than half of emotion measurements (14%–40% across studies) exhibited a unimodal symmetric distribution” (Haslbeck et al., 2023).

Suggests the presence of multiple states, and the need for time series models that accommodates multimodality, such as hidden Markov models.

Theory driven preference for HMMs

Current repeated measures and time series models aggregate across time points

However, we want to infer:

  • The dynamics over time in CAB crisis states
  • Distinct and ascending latent CAB crisis states over time
  • The composition of the latent, multidimensional CAB crisis states
  • How much patients differ in their dynamics over time
  • Patient specific CAB crisis trajectories over time

The hidden Markov model

HMM: probabilistic model to infer hidden states \(S_t \in (1, 2, ..., m)\) at

  • each time point \(t = 1, 2, ..., T\)
  • from the observed series \((y_{k1}, y_{k2}, ..., y_{kT})\)
  • for dependent variable \(k = 1, 2, ..., K\)

The hidden Markov model

We assume:

  • observation \(y_{kt}\) is generated by an underlying, latent state \(S_t\),
  • sequence of hidden states \(\{S_t : t = 1, 2, ..., T\}\) forms a Markov chain: probability of switching to the next state \(S_{t+1}\) only depends on the current state \(S_t\): \(P(S_{t_1} | S_t, S_{t-1}, ..., S_1 = P(S_{t+1} | S_t)\)

The hidden Markov model

Two sets of parameters:

  1. probability of transitioning between latent states \(\gamma_{ij}=P(S_{t+1} = j | S_t = i)\)
  2. probability of emitting an observation given the current latent state \(P(y_{kt} | S_t)\)

The hidden Markov model

Given example data, assume

  • Normal emission distribution: \(P(y_{kt} | S_t) \sim N(\mu_{ki}, \sigma_{ki}^2)\)
  • \(y_{k.}\) are conditionally independent given \(\{S_t : t = 1, 2, ..., T\}\) to accommodate multivariate data

HMM is suited for one sequence of data

Analyzing data from multiple individuals:

  • fit HMM to the data of each individual separately

  • fit one and the same HMM model to the data of all individuals

    • strong assumption: subjects do not differ with respect to HMM parameters
  • explain some of the differences using covariates (e.g., R package depmixS4)

  • Does not allow (quantification of) natural variation (i.e., heterogeneity) between individuals

  • Does not allow for individual specific dynamics over time

Extending the HMM to a multilevel framework

Including a multilevel framework for the HMM, enables simultaneous estimation:

  • group level parameter estimates
  • subject specific parameter estimates for each subject \(n\)

Multinomial logistic regression to estimate transition probabilities \(\gamma_{nij}\)

\(\gamma_{nij} = \frac{\text{exp}(\alpha_{nij})}{1 + \sum_{{j} = 2}^m \text{exp}(\alpha_{ni{j}})} \quad\) where \(\quad \alpha_{nij} = \bar{\alpha}_{ij} + \epsilon_{\left[\alpha\right]nij}\)

In addition, for the variable & state dependent emission Normal means:

\(\quad {\mu_{nki} = \bar{\mu}_{ki} + \epsilon_{\left[\mu\right]nki}}\)

Adopt a Bayesian approach

Four CAB based crisis states



CAB crisis state dynamics


Personalized crisis trajectories


Capturing crisis dynamics – conclusion

Pairing fine grained ESM data with multilevel HMM, we

  • Identified four distinct and ascending CAB based crisis states
  • Quantified and visualized the pattern of crisis at the group and patient individual level
  • Observed considerable variation between patients, both in remaining within and transitioning between CAB crisis states
  • Highlighted the need for a personalized method



Empirical example II

Nonverbal interpersonal interaction in patient-therapist dyads

Interaction in patient-therapist dyads

39 patient-therapist dyads

15 min annotated video recordings of therapy session

Possible research questions

  • Can we extract latent interpersonal interaction states over time?
  • What are the dynamics over time in interpersonal interaction states?
  • Do patient-therapist dyads differ in their dynamics over time?
  • Can this between dyad heterogeneity be explained by, for example, depression improvement?

The hidden Markov model

HMM: probabilistic model to infer hidden states \(S_t \in (1, 2, ..., m)\) at

  • each time point \(t = 1, 2, ..., T\)
  • from the observed series \((y_{k1}, y_{k2}, ..., y_{kT})\)
  • for dependent variable \(k = 1, 2, ..., K\)

The hidden Markov model

We assume:

  • observation \(y_{qt}\) is generated by an underlying, latent state \(S_t\),
  • sequence of hidden states \(\{S_t : t = 1, 2, ..., T\}\) forms a Markov chain: probability of switching to the next state \(S_{t+1}\) only depends on the current state \(S_t\): \(P(S_{t_1} | S_t, S_{t-1}, ..., S_1 = P(S_{t+1} | S_t)\)

The hidden Markov model

Two sets of parameters:

  1. probability of transitioning between latent states \(\gamma_{ij}=P(S_{t+1} = j | S_t = i)\)
  2. probability of emitting an observation given the current latent state \(P(y_{kt} | S_t)\)

The hidden Markov model

Given example data, assume

  • categorical emission distribution: \(P(y_{kt} | S_t) \sim Cat(\theta_{ki})\)
  • \(y_{k.}\) are conditionally independent given \(\{S_t : t = 1, 2, ..., T\}\) to accommodate multivariate data

Three nonverbal communication states

Dynamics in nonverbal communication


Explaining heterogeneity - including covariate(s)


Predicted nonverbal state dynamics


Predicted nonverbal state dynamics


Nonverbal communication state sequences


Interaction in patient-therapist dyads - conclusion



mHMMbayes

R package for multilevel hidden Markov models using Bayesian estimation

mHMMbayes

  • mHMMbayes fits multilevel hidden Markov models using Bayesian estimation
  • Allows for heterogeneity between subjects, while estimating one overall HMM
  • The model accommodates
    • continuous, categorical and count multivariate data
    • individual level covariates
    • missing data in the dependent variable(s)
  • The package also includes functions for automated plotting, simulating data, and obtaining the most likely hidden state sequence for each individual using the Viterbi algorithm.

mHMMbayes - Bayesian estimation

Advantages:

  • Bayesian methods yield many additional metrics, such as standard errors and credibility intervals, much harder to extract using frequentist methods.
  • Relatively easy to handle missing data (assuming MAR).
  • Allowing incorporation of prior knowledge (through prior distribution specification) accommodates smaller sample sizes.
  • Adds flexibility, making it relatively easy to extent to more complex models.

Disadvantages:

  • Label switching problem (more prominent in continuous and count data)
  • Solution: sensible starting values and weakly informative priors; (hyper-prior values for) \(\bar{\mu}_i\) or \(\bar{\lambda}_i\) ordered in a sensible manner given data and process

mHMMbayes - Metropolis within Gibbs sampler

  1. Obtain forward probabilities \(\alpha_{t(ni)} = Pr\big(O_1 = o_1, ..., O_t = o_t, S_t = i\big)\) for each subject using current values of subject specific model parameters.
  2. Sample hidden state sequences in backward manner using \(\alpha_{t(ni)}\) and \(\gamma_{nij}\).
  3. Given current subject-specific parameters, draw new group-level parameter estimates using Gibbs step
  4. Given
    • observed event sequence for each subject
    • sampled hidden state sequence for each subject
    • group-level parameter distributions
    • draw new subject-specific parameters using Random Walk (RW) Metropolis sampler
  5. Repeat step 1-4 a very large number of times

The mHMMbayes package: an example

39 patient-therapist dyads

15 min annotated video recordings of therapy session

Fitting a simple model – input

Fitting a 3 state multilevel hidden Markov model on the nonverbal data:

library(mHMMbayes)

## specifying general model properties:
 # number of states m
m <- 3
 # number of dependent variables
n_dep <- 6
 # number of output categories for each dependent var
q_emiss <- c(3, 2, 2, 3, 2, 2) 

Fitting a simple model – input

## specifying starting values
 # Start values of the TPM
start_TM <- diag(.9, m)
start_TM[lower.tri(start_TM) | upper.tri(start_TM)] <- .05

 # Start values for the emission distribution
start_EM <- list(matrix(c(0.70, 0.20, 0.10, # vocalizing therapist
                           0.05, 0.90, 0.05,
                           0.25, 0.70, 0.05), byrow = TRUE, nrow = m, ncol = q_emiss[1]), 
                  matrix(c(0.15, 0.85, # looking therapist
                           0.35, 0.65,
                           0.30, 0.70), byrow = TRUE, nrow = m, ncol = q_emiss[2]), 
                  matrix(c(0.80, 0.20, # leg therapist
                           0.80, 0.20,
                           0.80, 0.20), byrow = TRUE, nrow = m, ncol = q_emiss[3]) 
                  # continue emission probs here for the three other variables
                 ) 

Fitting a simple model – input

# Run a model without covariate(s) and default priors:
out_3st <- mHMM(s_data = MHMM_nonverbal,
                data_distr = "categorical",
                gen = list(m = m, n_dep = n_dep, q_emiss = q_emiss),
                start_val = c(list(start_TM), start_EMc),
                mcmc = list(J = J, burn_in = burn_in))

Model output – print()

out_3st
## Number of subjects: 39 
## 
## 5000 iterations used in the MCMC algorithm with a burn in of 2000 
## Average Log likelihood over all subjects: -2193.739 
## Average AIC over all subjects: 4447.478 
## 
## Number of states used: 3 
## 
## Number of dependent variables used: 6 
## 
## Type of dependent variable(s): categorical

Model output – summary()

summary(out_3st)
## State transition probability matrix 
##  (at the group level): 
##  
##              To state 1 To state 2 To state 3
## From state 1      0.942      0.019      0.040
## From state 2      0.021      0.928      0.051
## From state 3      0.071      0.103      0.826
## 
##  
## Emission distribution ( categorical ) for each of the dependent variables 
##  (at the group level): 
##  
## $T_Vocalizing
##         Category 1 Category 2 Category 3
## State 1      0.886      0.012      0.102
## State 2      0.018      0.976      0.006
## State 3      0.143      0.824      0.033
## 
## $T_Looking
##         Category 1 Category 2
## State 1      0.037      0.963
## State 2      0.307      0.693
## State 3      0.226      0.774
## 
## $T_Leg
##         Category 1 Category 2
## State 1      0.950      0.050
## State 2      0.947      0.054
## State 3      0.839      0.161
## 
## $P_Vocalizing
##         Category 1 Category 2 Category 3
## State 1      0.009      0.985      0.006
## State 2      0.884      0.005      0.111
## State 3      0.664      0.157      0.178
## 
## $P_Looking
##         Category 1 Category 2
## State 1      0.333      0.667
## State 2      0.045      0.955
## State 3      0.316      0.684
## 
## $P_Leg
##         Category 1 Category 2
## State 1      0.658      0.342
## State 2      0.998      0.002
## State 3      0.410      0.590

State composition – obtain_emiss()

emiss_pop <- obtain_emiss(out_3st)
emiss_pop
## $T_Vocalizing
##         Category 1 Category 2 Category 3
## State 1      0.886      0.012      0.102
## State 2      0.018      0.976      0.006
## State 3      0.143      0.824      0.033
## 
## $T_Looking
##         Category 1 Category 2
## State 1      0.037      0.963
## State 2      0.307      0.693
## State 3      0.226      0.774
## 
## $T_Leg
##         Category 1 Category 2
## State 1      0.950      0.050
## State 2      0.947      0.054
## State 3      0.839      0.161
## 
## $P_Vocalizing
##         Category 1 Category 2 Category 3
## State 1      0.009      0.985      0.006
## State 2      0.884      0.005      0.111
## State 3      0.664      0.157      0.178
## 
## $P_Looking
##         Category 1 Category 2
## State 1      0.333      0.667
## State 2      0.045      0.955
## State 3      0.316      0.684
## 
## $P_Leg
##         Category 1 Category 2
## State 1      0.658      0.342
## State 2      0.998      0.002
## State 3      0.410      0.590

State dynamics - obtain_gamma()

gamma_pop <- obtain_gamma(out_3st)
gamma_pop
##              To state 1 To state 2 To state 3
## From state 1      0.942      0.019      0.040
## From state 2      0.021      0.928      0.051
## From state 3      0.071      0.103      0.826

Specifying (weakly) informative-priors – gamma

Transition probability matrix Gamma (prior specification is optional):

prior_gamma(
  m,                  # number of states in the model
  gamma_mu0,          # hyper-prior mean values on the Multinomial logit intercepts
  gamma_K0 = NULL,    # number of hypothetical subjects gamma_mu0 is based on
  gamma_nu = NULL,    # degrees of freedom inverse Wishart hyper-prior on covariance 
  gamma_V = NULL,     # hyper-prior variance-covariance matrix of Inverse Wishart 
  n_xx_gamma = NULL   # number of level 2 covariates predicting gamma 
)


prob_to_int(
  prob_matrix         # complete transition probability matrix gamma
)

Specifying (weakly) informative-priors – emission

Categorical emission distribution (prior specification is optional):

prior_emiss_cat(
  gen,               # general model properties m, n_dep, and q_emiss
  emiss_mu0,         # hyper-prior mean values on the Multinomial logit intercepts
  emiss_K0 = NULL,   # number of hypothetical subjects emiss_mu0 is based on
  emiss_V = NULL,    # degrees of freedom inverse Wishart hyper-prior on covariance 
  emiss_nu = NULL,   # hyper-prior variance-covariance matrix of Inverse Wishart 
  n_xx_emiss = NULL  # number of level 2 covariates predicting emission distribution(s) 
)


prob_to_int(
  prob_matrix         # complete categorical emission probability matrix 
)

Specifying (weakly) informative-priors – emission

Continuous emission distribution (prior specification is MANDATORY):

prior_emiss_cont(
  gen,              # general model properties m and n_dep
  emiss_mu0,        # hyper-prior mean values of the Normal emission distribution 
  emiss_K0,         # number of hypothetical subjects emiss_mu0 is based on
  emiss_V,          # degrees of freedom inverse Gamma hyper-prior on BETWEEN subject variance 
  emiss_nu,         # hyper-prior variance of Inverse Gamma on BETWEEN subject variance 
  emiss_a0,         # shape values of IG hyper-prior on emission standard deviation^2 
  emiss_b0,         # scale values of IG hyper-prior on emission standard deviation^2 
  n_xx_emiss = NULL # number of level 2 covariates predicting emission distribution(s) 
)

More on IG hyper-prior on emission \(\sigma^2\)

Inverse Gamma hyper-prior on variance of Normal emission distribution:

References

  • J. Haslbeck, O. Ryan, & F. Dablander (2023). Multimodality and skewness in emotion time series. Emotion, 23(8), 2117–2141. Doi: 10.1037/emo0001218
  • W. W. Hale III, & E. Aarts (2023). Hidden Markov model detection of interpersonal interaction dynamics in predicting patient depression improvement in psychotherapy: Proof-of-concept study. Journal of Affective Disorders Reports, 14, 100635. Doi: 10.1016/j.jadr.2023.100635
  • S. Mildiner Moraga, F.M. Bos, B. Doornbos, R. Bruggeman, L. van der Krieke, E. Snippe, E. Aarts (2022). Evidence for severe mood instability in patients with bipolar disorder: Applying multilevel hiden Markov modelling to intensive longitudinal ecological momentary assessment data. PsyArXiv preprint. Doi: 10.31234/osf.io/egp82
  • Mildiner Moraga, S., & Aarts, E. (2023). Go Multivariate: Recommendations on Bayesian Multilevel Hidden Markov Models with Categorical Data. Multivariate Behavioral Research, 1-29. Doi: 10.1080/00273171.2023.2205392